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Abstract  k-i 

"J  In  many  application  areas,  particularly  in  the  biological  sciences,  there  is  the  need  to 
store  several  values  of  variables.  Given  a  finite  precison,  one  can  store  these  values  in 
explicit  cells,  refered  to  as  value  cells,  in  a  k-dimensional  Space  of  grain  N.  Typically,  the 
number  of  values  that  must  be  stored  is  a  very  small  fraction  of  the  total  number 
specified  by  the  grain  of  the  multidimensional  space.  This  leads  to  data  structuring  that 
reduces  the  number  of  explicit  cells  required  for  a  given  level  of  accuracy.  One  idea  is 
coarse  coding,  intersection  of  larger,  coarser  grained  cells.  Goarse  coding  has  been  show  n 
to  reduce  the  number  of  cells  required  by  a  factor  of  1/D*”  where  D  is  the  diameter  of 
the  coarse  cell  in  units  of  fine  grained  cells.  This  intuitively  appealing  idea  in  fact 
involves  many  subtle  tradeoffs  that  are  the  focus  of  this  paper.  Coarse  coding  is  show  n  to 
be  independant  of  the  isptrophy  of  the  cells  and  superior  to  simply  reducing  the  grain  of 
the  representation  space.  Loss  of  information  due  to  the  possibility  of  some  fine  grained 
cells  sharing  some  of  the  same  coarse  grained  cells  and  due  to  uncertainty  in  the  input 
and  translation  of  data  is  examined.  _  ~ 
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1.  Introduction 

The  encoding  and  access  of  values  in  a  multidimensional  space  is  an  integral  part  of 
many  computation  systems,  both  mechanical  and  biological.  For  example,  there  are  many 
inputs  available  to  the  visual  system,  such  as  position  in  space,  angle  of  rotation,  size, 
color,  depth,  velocity,  etc.  for  a  large  number  of  points  on  the  retina,  that  a  system  must 
be  able  to  access  frequently.  These  points  must  be  stored  with  some  great  degree  of  accu¬ 
racy,  as  well  (for  example,  the  eye  is  sensitive  to  positional  changes  of  5  arc  seconds). 
Thus  a  central  problem  is:  how  can  one  represent  a  discrete  set  of  m  points  in  a 
euclidean  space  of  dimension  k? 

One  way  of  encoding  a  set  of  values  is  simply  to  list  each  value  explicitly,  in  the 

form 


((coordinate, . coordinate^,  «-  point  1 

(coordinate, . coordinate^,  «-  point  2 

(coordinate,,  ... ,  coordinate^)  <-  point  m 

Unfortunately,  accessing  these  points  requires  time  O  (km),  which  would  make  real-time 
implementation  difficult.  An  alternative  is  to  represent  each  value  as  a  cell  in  a  discrete 
k-dimensional  space.  Values  can  be  accessed  in  0  (1)  time  by  a  computer  with  parallel 
architecture  or  a  biological  connectionist  network  (Feldman  and  Ballard[l]).  This  strategy 
also  has  drawbacks,  however,  the  most  important  being: 

(1)  The  finite  number  of  discrete  values  (the  grain  ),  places  limits  on  the  accuracy  with 
which  values  can  be  represented.  This  has  important  consequences  for  biological 
computation  systems  in  particular,  for  if  there  is  a  limit  on  how  many  different 
values  can  be  encoded,  then  that  limit  applies  to  the  amount  of  processes  that  can  be 
run  in  parallel  in  the  brain  (which  consists  of  value  cells  called  neurons).  To  com¬ 
pensate  for  this,  a  very  fine  grain  can  be  used. 

(2)  This  strategy  requires  Nk  discrete  units  for  representation  (where  N  is  the  grain). 
This  problem  is  the  more  important  of  the  two,  as  both  mechanical  and  biological 
systems  have  limits  on  the  amount  of  storage  available.  For  example,  100  discrete 
values  over  each  of  10  parameters  is  probably  less  than  what  the  human  brain  uses 
for  vision,  but  even  this  would  require  1020  units  of  storage,  which  is  far  greater  than 
the  number  of  neurons  in  the  human  brain. 

1.1.  Overlapping  Cells 

One  method  of  reducing  the  amount  of  cells  involves  overlapping  cells.  Overlapping 
receptive  fields  are  a  constant  phenomenum  in  biology  but  only  recently  has  work  been 
done  analyzing  the  usefulness  of  this  representation  for  other  applications.  General  conse¬ 
quences  of  computing  with  this  kind  of  method  are  explored  in  Baliard[2];  the  first 
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analysis  of  their  efficiency  is  given  by  Hinton[3J. 

This  method  decomposes  a  space  of  size  Nk  into  D  coarser  grained  spaces  of  size 
(N/D)k  each  by  having  each  unit  in  the  coarse  spaces  represent  a  k-dimensional  cube  D 
units  on  a  side  in  the  Nk  fine  space.  The  divisions  in  each  of  the  coarse  spaces  are  chosen 
in  such  a  way  that  each  point  in  fine  spaces  has  a  unique  representation  in  the  set  of 
points  that  it  corresponds  to  in  the  set  of  D  coarse  spaces.  This  is  done  by  offsetting  each 
of  the  D  spaces  by  one  diagonal  unit.  That  is,  the  d/*[th/*]  coarse  array  is  offset  an 

amount  (d-1 . d-1).  Then,  given  the  fine  grained  value,  the  D  coarse  grained  cells  can 

be  determined  by  rounding  with  appropriate  offsets: 

(x0,  y0)  =  x/D,  y/D 

(xp  yj)  =  (x  +  1)/D,  (y  +  1)/D 


(xD-r  Vd-i)  =  (x  +  D-l)/D,(y  +  D-l)/D 

Figure  I 

Recovery  of  data  in  coarse  coding  (for  k  =  2  and  D  =  3) 

The  number  of  units  required  for  this  representation  is  Nk/Dk  l. 

In  principle  the  points  in  fine  space  need  only  be  represented  implicitly  by  the 
coarse  units.  That  is,  the  coarse  arrays  behave  as  if  fine  points  were  represented.  Points  in 
the  fine  space  may  be  recovered  by  allowing  the  points  in  each  of  the  coarse  spaces  to 
vote  into  each  of  the  D  points  in  the  fine  space  that  it  represents.  The  votes  from  the 
various  coarse  spaces  will  overlap  and  the  original  point  (and  that  point  only)  will  recieve 
D  votes.  Other  points  will  receive  less  than  D  votes.  This  method  of  specifying  the  fine 
value  we  term  the  intersection  method. 

Because  of  the  l/D1**1  space  savings,  the  idea  of  overlapping  cells  is  attractive,  but 
there  are  a  number  of  interesting  questions. 

1)  Uniform  overlapping  fields,  where  proportionate  cell  sizes  are  all  similar  (isotropic 
scaling)  is  one  possible  representation  of  multi-dimensional  space,  but  there  is  also 
the  possibility  of  subspaces  (anisotrpic  scaling).  These  alternatives  are  examined  (in 
section  3),  and  are  shown  to  be  similar  to  isotropic  scaling. 

2)  If  the  degree  of  overlap  is  too  large  false  signallings  can  result.  This  is  known  as  the 
illusory  conjunction  problem.  Since  the  severity  of  this  problem  depends  on  the 
statistics  of  the  signal,  we  explore  the  extent  of  this  problem  through  simulations. 

3)  Finally,  we  distinguish  between  logical  (or  discrete)  noise,  which  results  from 
processes  interfering  by  occupying  too  many  points  too  near  to  each  other  in  the 
space,  and  arithmetic  (or  continuous)  noise,  in  which  errors  in  the  input  and/or 
translation  mechanisms  cause  votes  for  a  single  process  to  be  spread  over  many 
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contiguous  points  instead  of  a  single  point  in  the  space. 

2.  Overlapping  Cells 

2.1.  Overlapping  Cells  and  Noise 

Overlapping  fields  are  useful  when  there  is  a  good  deal  of  noise  associated  with  the 
input,  as  noise  can  dramatically  effect  the  usefulness  of  non-overlapping  cells.  Consider 
accumulating  evidence  for  a  single  process  where  the  individual  measurements  are  subject 
to  some  probability  of  error.  Problems  will  result  if  the  noise  variance  is  large  compared 
to  the  cell  size.  Even  if  it  were  not,  if  the  exact  position  of  the  process  falls  on  the  border¬ 
line  between  two  cells,  then  some  values  will  fall  into  one  cell  and  others  will  fall  into  the 
other.  In  either  case,  the  estimates  for  the  correct  position  in  the  parameter  space  are  scat¬ 
tered  over  nearby  cells. 


Figure  2 

Noise  in  estimating  cells 

One  could  try  to  correct  these  problems  by  estimating  the  correct  cell  as  the  average  of 
contiguous  cells  over  a  noise  threshold  T.  This  may  not  work,  however,  if  the  grain  is  too 
fine,  as  the  data  may  be  spread  such  that  not  all  points  are  contiguous.  This  would  cause 
the  mechanism  to  interpret  it  as  several  processes,  none  of  which  having  enough  evidence 
to  be  important.  Overlapping  cells  seem  to  solve  these  problems,  as  each  cell  sees  more 
input.  Since  the  evidence  that  a  fine  grained  point  receives  from  this  representation  is 
very  roughly  proportional  to  the  inverse  of  the  distance  from  points  close  to  it  (as  those 
are  more  likely  to  fall  into  more  cells  that  also  represent  that  particular  point),  overlap¬ 
ping  cells  do  a  good  job  of  compensating  for  both  the  error  spread  and  borderline  prob¬ 
lems.  For  more  precise  mathematical  details,  see  Section  4. 

2.1.1.  Ambiguity  in  Overlapping  Cells 

There  is,  of  course,  some  loss  of  information  associated  with  any  reduction  in  the 
size  of  the  representation.  In  this  case,  if  enough  of  the  overlapping  cells  that  represent  a 
point  in  the  fine  grained  space  have  been  activated  by  other  points,  then  there  is  no  way 
of  knowing  whether  that  point  should  be  on  or  off. 

Figure  3 

Example  of  ambiguity  in  coarse  coding 
points  I  and  2  activate  all  overlapping  cells  of  point  3 

This  kind  of  error  could  result  from  noise,  but  is  more  likely  to  be  the  result  of  two 
processes  being  too  close  together  and  sharing  many  of  the  same  overlapping  cells.  The 
mechanism  would  not  be  able  to  scperate  the  two  processes  with  any  degree  of  accuracy . 

Determining  the  probability  of  error  is  very  difficult,  as  determining  the  probability 
that  n  irregular  shapes  do  not  overlap  is  an  unsolved  problem  for  greater  than  one 


dimension.  Instead,  a  large  number  of  simulations  were  run,  the  results  of  which  are 
given  in  Table  1.  The  test  runs  involved  N  s_of  4,  8,  16,  32,  64,  and  128,  k's  of  2  -  8,  and 
D's  from  f  N  to  N/2  by  multiples  of  f  2.  Since  determining  whether  error  occured 
requires  reconstructing  the  original  Nk  space  (and  the  computer  these  runs  were  made  on 
simply  does  not  have  that  much  memory),  error  was  considered  to  have  occured  when 
two  points  shared  any  of  the  same  overlapping  cehs.  The  numbers  in  the  table  represent 
the  logarithm  (base  2)  of  the  minimum  number  of  points  needed  to  cause  at  least  a  10% 
chance  of  error.  This  was  obtained  by  making  50  runs  and  increasing  the  number  of 
points  by  a  factor  of  two  until  at  least  five  of  the  runs  reported  error. 

3.  Subspaces 

3.1.  Definition 

Another  way  to  represent  an  Nk  space  is  to  split  a  k-dimensional  spare  into  its  k 
component  1-dimensional  spaces  by  projecting  the  data  into  its  various  parameters.  This 
is,  in  effect,  the  ultimate  anisotropic  cell  representaion  as  each  cell  has  the  size  of  the 
space  in  one  dimension  and  size  1  in  all  of  the  rest.  An  example  is  shown  in  Figure  4, 
where  points  in  an  XY  plane  are  projected  onto  X  and  Y  axes.  This  subspace  representa¬ 
tion  requires  only  k  times  the  k*  root  of  the  space  required  by  the  original  representa¬ 
tion,  a  substantial  savings. 

Figure  4 

Two-dimensional  subspace  example 

Again,  it  is  impossible  to  reduce  the  size  of  a  representation  without  some  informa¬ 
tion  loss.  In  this  case,  we  no  longer  know  which  points  in  the  subspaces  correspond  to 
each  other.  Thus,  when  reconstructing  the  original  m  points  from  this  representation,  we 
would  have  no  choice  but  to  associate  all  of  the  points  in  the  k  subspaces  with  each 
other.  This  would  result  in  mk  points,  or  mk  -  m  extraneous  associations,  as  shown  in  Fig¬ 
ure  5.  This  is  clearly  unacceptable. 

Figure  5 

Ambiguity  of  multiple  data  in  subspace 

3.2.  Coarse-fine  connections 

This  problem  of  ambiguity  could  be  solved  by  interconnecting  all  of  the  points  in 
the  two  subspaces  by  intermediate  units,  but  that  would  result  in  the  original  k- 
dimensional  space.  Instead,  the  solution  is  to  connect  large  areas  between  the  spaces. 
These  coarse  connections  could  be  represented  as  an  k-dimensional  arra\,  but  with  a 
much  coarser  grain  than  the  subspaces  that  it  connects.  Points  in  this  arra>  would 
represent  a  connection  between  points  that  lie  in  the  areas  of  the  two  subspaces  (as  parti¬ 
tioned  by  the  coarse  divisions).  This  new  representation  would  require  k\  +  (\/D)k 


units,  where  is  the  size  of  the  k-dimensional  coarse  connection  array. 

Figure  6 

Recovery  of  data  in  coarse-fine  subspace 

Of  course,  ambiguity  is  still  possible.  If  two  connections  occupy  the  same  unit  in  the 
coarse  connection  space,  there  is  still  no  way  to  tell  which  values  of  the  parameters  are 
associated  with  which  for  those  values  connected  to  that  point,  but  the  chances  of  this 
happening  are  greatly  reduced.  To  determine  the  expected  error,  note  that  this  is 
mathematically  identical  to  the  number  of  urns  with  two  or  more  recing  a  number  of 
balls  in  a  number  of  urns  at  random  and  defining  the  number  of  errors  to  be  the  number 
of  urns  with  two  or  more  balls  in  them.  The  expected  number  of  errors,  then,  given  m 
points  placed  in  (N/D)k  discrete  units  in  the  coarse  connection  space,  is 

(D/N)k(mC2Xl  *  (D/N)kr2 

where  C  =  m!/n!(m-n)! 

as  given  in  Johnson  and  Kotz[4], 

3.3.  Comparison  with  coarse  coding 

For  examples  of  how  many  points  this  representation  can  reasonably  store,  in  com¬ 
parison  to  the  results  of  overlapping  coding,  see  Table  2  which  contains  results  for  the 
same  parameters  as  in  Table  1  computed  with  the  above  equation.  At  first  glance  the 
results  look  very  similar.  Coarse-fine  coding  is  slightly  more  efficient,  but  only  by  a  factor 
of  four  at  the  extreme  of  the  range  measured.  This  is  to  be  expected,  perhaps,  as  there 
will  always  be  a  certain  amount  of  information  lost  when  the  size  of  the  representation 
space  is  reduced,  and  the  loss  is  probably  far  more  dependant  on  that  reduction  than  on 
the  method  of  representation.  If  this  is  the  case,  then  if  we  use  coarse  coding  (for  which 
no  simple  error  model  exists)  we  can  estimate  the  expected  number  of  errors  using  the 
coarse-fine  coding  model  above. 

4.  Continuous  Noise 

4.1.  Sources  of  Continuous  Gaussian  Noise 

So  far  we  have  been  looking  at  discrete  noise,  interference  produced  by  discrete 
processes  running  in  the  same  space.  If  the  cells  themselves  represent  discrete  values 
(such  as  high  level  concepts  like  "apple"  and  "orange"),  then  we  would  expect  one  cell 
and  only  one  ceil  to  be  activated  by  some  input.  If  the  cells  represent  continuous  values 
such  as  the  position  of  an  object  in  space  or  the  frequencies  corresponding  to  its  color, 
however,  then  we  would  expect  a  certain  amount  of  uncertainty  concerning  which  cells 
are  to  be  activated  corresponding  to  the  uncertainty  in  measuring  these  quantities  in  the 
first  place,  and  this  continuous  noise  is  additional  to,  and  must  be  modeled  differently 
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I  from  the  discrete  noise  described  above.  For  example,  most  images  in  the  real  visual 

|  world  have  a  certain  amount  of  "blurring"  associated  with  them,  where  the  measurement 

|  process  can  be  modelled  as  an  ideal  value  corrupted  by  noise.  When  these  images  are 

translated  to  a  discrete  encoding,  they  may  activate  cells  close  to  but  not  exactly  the 
correct  cell  for  the  process.  In  general,  such  random  noise  from  the  real  world  is  gaussian, 
so  if  we  coarsely  encoded  a  gaussian  distribution  of  votes  over  a  set  of  fine  cells  (in  this 
example  the  retina)  we  would  expect  the  votes  for  the  coarse  cells  to  exhibit  that  distribu¬ 
tion. 

4.2.  Roundoff  Error 

Another  source  of  error  occurs  when  translating  from  one  ecoding  scheme  to 
another.  In  figure  5  we  are  translating  space  parameters  (x,  y)  to  line  partameters  (r,  T). 
In  this  example,  a  cell  in  rT  space  may  correspond  to  several  cells  in  xy  space,  but  since 
each  cell  in  xy  space  represents  a  region  instead  of  a  single  point  it  may  not  be  possible  to 
partition  these  regions  in  such  a  way  that  any  two  regions  in  xy  space  corresponding  to 
two  cells  in  xy  space  overlap  exactly  when  translated  into  the  same  cell  in  r  space.  In  a 
case  like  this  a  cell  in  one  space  may  not  correspond  exactly  to  a  cell  in  another.  It  may 
correspond  mostly  to  one  cell,  but  partly  to  others  depending  on  how  the  regions  are 
translated.  To  compensate  for  this  we  could  connect  a  cell  in  one  space  to  several  in  the 
same  region  of  another  space  (instead  of  just  one),  and  give  votes  to  the  cells  in  such  a 
way  that  they  make  up  a  gaussian  ditribution  centered  around  the  translation  of  the 
region  of  the  cell  in  the  first  space.  Thus,  if  a  region  of  a  cell  in  one  space  lies  on  the 
"edge"  of  the  corresponding  cell  in  the  other  space,  we  would  give  that  edge  more 
"weight"  by  giving  votes  to  adjoining  cells  on  that  edge.  This  can  be  done  by  1)  comput¬ 
ing  a  gaussian  curve  based  at  the  exact  point  that  the  data  translates  to  and  2)  giving  each 
cell  a  vote  corresponding  to  the  integral  of  that  curve  over  the  volume  of  the  cell. 

I  Figure  7 

Continuous  noise  in  translation 


4.3.  Setting  Thresholds  to  Eliminate  Continuous  Noise 

As  mentioned  earlier,  the  acuity,  or  overall  noise  resistance  of  the  system  may  be 
greatly  improved  by  coarse  coding.  Instead  of  having  to  rely  on  a  number  of  votes  exactly 
choosing  the  correct  cell,  we  can  now  rely  on  the  effects  of  "near  misses".  More  precisely, 
we  can  give  coarse  cells  a  certain  threshold  t,  and  consider  a  fine  cell  to  be  activated  if  all 
of  the  coarse  cells  that  represent  it  have  reached  some  given  activation  threshold ,  as 
opposed  to  setting  a  threshold  for  the  individual  fine  cells  themselves.  In  order  to  elim¬ 
inate  all  noise,  the  threshold  must  be  chosen  large  enough  so  that  all  coarse  cells 
corresponding  to  the  correct  fine  cell  (which  we  are  assuming  is  at  the  center  of  the  gaus¬ 
sian  noise  distribution)  are  activated,  but  small  enough  so  that  none  others  are  activated. 
Of  these  two  goals  the  first  is  far  more  important,  as  it  is  far  better  for  a  process  to  ha\e 
some  noise  associated  with  it  that  is  is  for  the  process  to  disapear  altogether. 
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First,  we  will  determine  the  lower  bound  necessary  to  activate  the  correct  cell. 

Given  a  standard  gaussian  correct  cell-centered  noise  model,  the  expected  number  of 
overall  votes  needed  to  activate  all  coarse  cells  needed  will  depend  on  the  expected 
number  of  votes  needed  to  activate  the  cell  farthest  from  the  center  of  the  curve  (the  cell 
expected  to  recieve  the  least  number  of  votes).  In  the  worst  possible  case  this  will  be  a 
coarse  cell  in  which  the  correct  cell  lies  in  one  of  the  comers. 

Figure  8 

Worst  case  coarse  cell  arrangement  for  cell  activation 

Given  a  gaussian  vote  distribution  defined  by  Ae‘r2/<r  we  can  define  the  expected  number 
of  votes  recieved  by  a  coarse  cell  to  be  the  sum  of  the  distribution  over  the  area  of  the 
coarse  cell.  In  the  case  of  a  "comer  cell",  this  would  be  from  -1/2  to  D-l/2  in  all  k 
dimensions,  or 

D-l/2 

/ / f ...  Ae'<xi:+x2  +  dxl  dx2  ...  dxk 

-1/2 

i 

i 

which  is 


r,  =  A(2<W  7r(Erf(l/2cr)  +  Erf([(2D-l)/2a))k 

where  Erfix)  is  the  error  function,  which  has  derivative  2e'x  / ^  'n- 

The  next  step  is  to  find  an  upper  bound  on  the  number  of  votes  that  a  coarse  cell 
not  overlapping  the  correct  fine  cell  recieves.  If  any  of  these  cells  are  activeted  then  other 
fine  cells  besides  the  correct  one  will  be  indicated,  so  our  threshold  must  be  great  enough 
so  that  cell  is  not  activated.  Unfortunately,  the  exact  pattern  of  coarse  cells  that  determine 
a  fin^  ell  is  not  fixed,  as  shown  in  figure  9: 

Figure  9 

Different  ways  that  coarse  cells  can  combine  to  specify’ fine  cells 

so  the  pattern  of  coarse  that  determined  the  worst  case  "comer  cell"  above  may  not  be 
the  same  pattern  that  determines  the  worst  case  cell  not  overlapping  the  correct  fine  cell. 
That  case  occurs  when  the  correct  ceil  is  adjecent  to  a  coarse  ceil  in  one  dimension  and  in 
the  center  of  that  cell  in  the  other  k-1  dimensions,  as  in  figure  10. 

Figure  10 

Worst  case  for  non- activated  coarse  cell 

Since  this  "side  cell"  is  actually  closer  to  the  center  of  the  gaussian  curve  than  the  corner 
cell  in  most  of  the  dimensions,  it  is  clear  that  it  will  almost  certainly  recieve  more  votes. 


This  means  that  it  is  impossible  to  choose  a  threshold  that  activates  all  correct  coarse  cells 
and  none  of  the  incorrect  ones.  Since  we  must  activate  all  correct  coarse  cells,  we  will 
have  to  accept  some  incorrect  cells,  and  some  uncertainty  concerning  which  of  the 
corresponding  fine  cells  is  the  correct  one.  This  can  be  compensated  for  by  averaging  the 
values  of  a  local  group  of  activated  cells  in  order  to  estimate  the  correct  value. 

Thus  it  appears  that  the  best  way  to  correct  this  problem  is  to  find  a  way  to  reduce 
<r,  the  size  of  the  noise  spread.  One  way  that  the  human  visual  system  does  this  is  to  use 
the  time  dimension  to  augment  the  k-dimensional  space.  Cells  further  away  from  a  point 
in  the  space  are  sampled  less  often  than  those  closer  to  the  point  when  determining  the 
number  of  votes  for  a  point.  This  means  that  a  cell  would  not  recieve  as  many  votes  from 
an  input  that  lies  in  one  of  its  comers  as  it  would  from  one  that  lies  in  its  center  over  a 
period  of  time. 

Another  solution  would  be  to  augment  these  logical  receptive  fields  with  arithmetic 
receptive  fields  that  would  determine  the  average  position  of  activation  over  some  range 
of  cells  cel^.  This  is  done  by  calculating 

value  =  2  v(  X  val 

i 


i 


where  vv  is  the  number  of  votes  for  cellt  and  val^  is  the  numerical  value  that  cellt 
represents,  for  each  cell  with  greater  than  some  fixed  threshold  T. 

5.  Vernier  Representation 

Another  possible  use  of  these  strategies  is  as  a  vernier  technique  [5],  If  one  wanted  to 
store  a  number  of  exact  values  (more  exact  than  the  grain  will  allow  ),  and  one  wanted 
parallel  access  to  the  approximate  value  of  that  value,  then  the  approximate  value  could 
be  stored  in  a  very  coarsely  grained  space,  with  a  connection  from  that  cell  to  a  sequen¬ 
tial  access  representation  of  the  exact  value. 

Figure  11 

Vernier  representation 

6.  Conclusions 

Overlapping  coding  is  a  method  of  representing  high-dimensional  values  in  a  rela¬ 
tively  small  space  where  values  may  be  recovered  in  parallel  time.  As  may  be  expected, 
reducing  the  size  of  the  representation  space  causes  a  certain  amount  of  information  to  be 
lost,  and  this  loss  is  more  or  less  indepentant  of  the  isotrophy  of  the  representation. 


When  choosing  the  coarseness  of  the  grain  (D)  to  use  with  these  methods,  the  user 
must  balance  the  costs  of  added  space  against  the  costs  of  error.  If  a  relatively  noise- 
resistant  system  such  as  the  Hough  transform  is  used,  then  very  large  D  may  be  chosen 
and  a  considerable  amount  of  space  may  be  saved.  (Such  a  system  has  been  imple¬ 
mented  for  two-dimensional  image  recognition.  In  that  case  D  was  set  to  V  N,  thus  sav¬ 
ing  space  by  a  factor  of  V  N,  without  causing  any  significant  loss  of  accuracy.) 
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Figure  1 

Recovery  of  data  in  coarse  coding  (for  k  =  2  and  D  =  3) 
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two-dimensional  XY  space 
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two-dimensional  XY  space 


Figure  5 

Ambiguity  of  multiple  data  in  subspaces 
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Figure  6 

Recovery  of  data  in  coarse-fme  subspace 

Associations  between  points  in  X  and  Y  subspaces 
must  lie  within  an  activated  region  in  the  coarse-fine  subspace 


Figure  7 

'ontinuous  noise  in  translation 

An  edge  xy0  might  map  into  the  penpherary  of  the  r,T  cell  Weights  can 

compensate  for  this. 


Figure  9 

Different  ways  that  coarse  cells  can  combine  to  specify  fine  cells 
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